1. Field of Invention
The present invention relates, in general, to the testing of software and, in particular, to a system that maximizes reusability of testing components in both manual and automated testing. More particularly, the present invention goes beyond current data driven and keyword driven testing approaches by making test data fully reusable. This, in turn, makes other testing assets more reusable and serves to lower the cost of developing and maintaining test assets.
2. Description of Related Art
Fundamentally, software testing involves exercising a system, including software under test, with a known initial state and predefined input. The actual results of the system are compared with predefined expected results. Thorough testing requires exercising a system with many variations in both the sequence of actions performed and test data (i.e., predefined input or expected results). Software testing, especially manual testing, can be both tedious and labor intensive. Moreover, the nature of software development is that a system will need to be re-tested every time the system is modified in order to verify that modifications work as intended and do not introduce unforeseen defects in what is known as regression testing. Maintaining test cases can be expensive—much like maintaining a software system.
Automating testing execution to reduce test execution time and reusing testing components to cut test planning and maintenance costs are two of the most common strategies to reduce software testing costs. Data driven testing and keyword testing are two approaches for accomplishing this.
Data driven testing involves reading a table of data, such as from a spreadsheet, where each table row contains a record of test data (i.e., the predefined input and expected results). The advantages of data driven testing are the ability to easily add new conditions to the table and the ability to specify the test data separate from the test. Data driven testing enables a limited form of reuse. While the test can be reused with the different data conditions in the table, the table itself is closely bound to the test case and therefore the test cannot be easily reused with a different table of test data. Moreover, the test data must be customized to the specific test to be conducted and therefore test data (i.e., individual records of the table) are not easily reused. This, in turn, limits the ability to reuse the test data with different tests.
In contrast, keyword driven testing uses a keyword to represent a specific action (e.g., user login, data entry, check box actions, etc.). A test driver is written to read the keywords and execute test scripts associated with the keyword. Data is usually embedded in the test or in an external table as in data driven testing. Keyword driven testing provides a high level of reuse for the keywords/scripts, but, like data driven testing, the data is bound to the specific test being performed, so the test is not easily used with different data, and the data is not easily used with other tests.
While both data driven and keyword testing achieve some level of reuse, neither approach is designed to reuse the test data itself. Testing tools designed to be a repository of test cases, when they don't ignore data altogether, typically either imbed test data in the test case or in external datasheets. These approaches force test data to be designed around a specific test, binding the test data to the specific test and thus limiting the reusability of the test and making the reusability of the test data impractical. Another issue with organizing test data around individual tests is that it can be difficult to determine if a given test condition is, in fact, tested. When test data is spread over numerous spreadsheets or other data sources and imbedded as parameters in tests, it becomes impractical to determine if a given condition is tested and, if so, where it is tested. This non-reusable test data is thus difficult to maintain and works to lower the reusability of the associated tests and scripts.